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pencil  measures  for  predicting  job  performance  abilities  was  the  objective  of  a 
series  of  studies.  This  report  covers  the  initial  test  development  and  analysis 

r  search. 

Eight  computerized  tests  were  constructed  to  measure  five  personal  attributes 
identified  in  previous  research  as  being  important  for  job  performance.  The 
experimental  battery  also  contained  nine  previously  developed  tests.  The 
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relationships  were  analyzed  in  conjunction  with  operational  written  test  and 
biographical  variables.  Correlational  and  factor  analyses  were  carried  out. 

Computerized  tests  were  particularly  important  for  measuring  sequential 
information  processing,  movement  detection  and  short-term  memory  skills.  They 
offered  no  advantage  over  paper-and -pencil  measures  of  perceptual  speed,  and 
findings  relative  to  perceptual  closure  were  ambiguous. 

Computerized  and  paper-and -pencil  tests  of  perceptual  closure  apparently 
measured  different  abilities. 

The  two  separate,  short-term  memory  abilities  found  corresponded  to 
faculties  for  high  associational  and  low  associational  (rote)  stimuli. 
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FOREWORD 


This  research  was  performed  under  Task  Area  Number  RR042-04-01 
(Job  Element  Approach  to  the  Validation  of  Perceptual  Measures)  and 
Work  Unit  Number  NR  150-335.  It  was  supported  by  Personnel  and 
Training  Research  Programs  of  the  Office  of  Naval  Research. 

The  research  was  carried  out  to  investigate  a  proposal 
originally  developed  by  Dr.  Rebecca  Bryson,  who,  at  the  time, 
was  assigned  to  the  then-existing  Personnel  and  Training  Research 
Laboratory,  San  Diego.  The  author  wishes  to  thank  Dr.  Bryson  for 
her  many  suggestions  concerning  the  approach  and  methodology  of 
the  study. 
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SUMMARY 


Problem 


Considerable  evidence  exists  that  the  low  validities  frequently  found 
for  paper-and-pencil  tests  against  job  performance  criteria  may  be  conse¬ 
quences  of  ability  requirements  for  job  performance  which  are  too  varied 
to  be  completely  measurable  by  written  tests.  Thus  other  modes  of  test 
administration  may  provide  useful  measures  for  some  types  of  jobs.  Com¬ 
puterized  testing  permits  the  assessment  of  many  additional  skill  charac¬ 
teristics  beyond  those  measurable  in  most  paper-and-pencil  tests,  of  which 
some  abilities  may  be  useful  for  classification  and  assignment  purposes. 

This  report  summarizes  the  first  of  two  studies  intended  to  evaluate  com¬ 
puterized  tests  as  predictors  of  on-job  performance.  It  covers  the  sta¬ 
tistical  characteristics  and  factor  structure  of  a  set  of  experimental 
and  operational  tests. 

Approach 

Eight  computerized  tests  were  constructed  to  measure  five  personal 
attributes  identified  in  previous  research  as  being  important  for  job 
performance.  The  experimental  battery  also  included  previously  developed 
tests,  a  number  of  which  were  paper-and-pencil  tests  from  factor  reference 
batteries.  The  17  experimental  tests  were  administered  to  385  enlisted 
personnel  who  were  in  training  for  the  Electrician's  Mate,  Personnelman, 
or  Sonar  Technician  ratings,  or  were  unassigned  recruits.  Scores  were 
analyzed  in  conjunction  with  operational  biographical  and  test  variables. 

Findings  and  Conclusions 

1.  Computerized  tests  were  particularly  important  for  measuring  sequen¬ 
tial  information  processing,  movement  detection,  and  short  term  memory 
skills.  They  offered  no  advantage  over  paper-and-pencil  measures  of  per¬ 
ceptual  speed,  and  findings  relative  to  perceptual  closure  were  ambiguous. 
(Pages  12,  13,  and  14.) 

2.  Despite  apparent  similarity  of  format,  computerized  and  paper-and- 
pencil  tests  of  perceptual  closure  apparently  measured  different  abilities. 
Additional  research  should  be  carried  out  to  investigate  more  specifically 
the  characteristics  of  these  closure  tests.  (Pages  11,  12,  15,  and  16.) 

3.  Two  separate,  short  term  memory  abilities  were  found,  corresponding 
to  faculties  for  high  associational  and  for  low  associational  (rote)  stimuli. 
(Pages  14  and  15.) 

4.  Considerable  diversity  of  coverage  existed  among  tests,  even  tests 
ostensibly  measuring  the  same  personal  attributes.  Measures  of  movement 
detection  and  dealing  with  concepts/information  were  more  diffuse  than 
those  of  short  term  memory  and  perceptual  speed.  (Pages  12  and  13.) 
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AN  EVALUATION  OF  COMPUTERIZED  TESTS  AS  PREDICTORS 
OF  JOB  PERFORMANCE  IN  THREE  NAVY  RATINGS: 

I.  DEVELOPMENT  OF  THE  INSTRUMENTS 


BACKGROUND 


Despite  more  than  a  half  century  of  development  and  improvement  of 
tests  used  for  the  selection  and  assignment  of  personnel  to  jobs,  the 
level  of  test  validity  achieved  remains  disappointingly  low.  Rundquist 
(1967),  among  others,  has  pointed  out  that  there  is  no  evidence  that 
selection  techniques  have  improved  during  the  past  six  decades. 

Furthermore,  as  Ghiselli  (1959)  has  noted,  there  is  frequently  little 
consistency  in  validity  coefficients  even  for  the  same  type  of  job  from 
company  to  company.  Thus  in  one  series  of  studies  he  found  that  validity 
coefficients  of  intelligence  tests  for  general  clerical  positions  ranged 
from  -.40  to  +.80,  with  the  middle  50  percent  of  the  correlation  coeffi¬ 
cients  covering  a  range  of  50  correlation  points.  Similar  variations  in 
correlation  coefficients  were  observed  for  other  test-occupation  combi¬ 
nations. 

Ghiselli  suggested  that  part  of  the  difficulty  in  achieving  consist¬ 
ent  relationships  can  be  attributed  to  differences  in  job  content  from 
company  to  company.  Other  probable  causes  include  differences  among 
supervisors  in  rating  standards  and  biases,  e.g.,  halo  effects.  Rundquist 
attributed  similar  findings  to  the  multidimensional  character  of  abilities 
required  for  performance  on  jobs.  Because  of  the  lack  of  one-to-one  cor¬ 
respondences  between  abilities  and  performance  on  jobs,  deficiencies  in 
one  ability  may  frequently  be  counteracted  by  strengths  in  other  abilities. 
For  example,  good  performance  on  a  memory  task  might  result  from  a  superior 
memory,  or  efficient  grouping  techniques,  or  from  combinations  of  these 
abilities. 

These  considerations  suggest  that  despite  extensive  test-job  perform¬ 
ance  validation  research,  personnel  experts  at  the  present  time  have  only 
a  hazy  understanding  of  the  predictive  relationships  between  tests  and 
subsequent  job  performance.  Since  most  test  validations  have  utilized 
global  performance  ratings  as  criteria,  there  is  little  understanding  of 
which  elements  within  the  total  job  are  predictable  and  which  are  unpre¬ 
dictable.  Furthermore,  the  types  of  tests  which  are  predictive  of  indi¬ 
vidual  job  elements  are  also  essentially  unknown. 

Studying  the  predictive  relationships  of  tests  to  individual  job 
elements  would  shed  light  on  these  relationships.  In  addition,  studying 
the  test-job  element  predictive  relationships  might  provide  a  means  of 
counteracting  the  well  known  deficiencies  in  the  global  criterion  rating 
resulting  from  halo  effect  and  varying  supervisory  standards. 

Global  ratings  have  long  been  known  to  be  unduly  influenced  by  halo 
effects.  By  confining  the  attention  of  raters  to  more  narrowly  defined 
and  highly  specifiable  criteria — such  as  are  contained  in  the  individual 
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job  elements — the  ratings  are  more  likely  than  are  global  ratings  to  con¬ 
form  to  actual  performance  differences  on  the  criteria.  In  addition, 
differences  in  supervisory  perceptions  of  the  importance  of  specific  job 
elements  for  global  performance  can  be  made  manifest  in  a  study  focussed 
on  predictability  of  job  elements.  By  forming  the  global  evaluation  as 
a  composite  of  the  relatively  accurate  job  element  evaluations,  some  of 
the  effects  of  halo  should  be  counteracted. 

Furthermore,  if  reliable  relationships  between  test  scores  and  spe¬ 
cific  job  elements  can  be  found,  it  may  be  possible  to  build  more  valid 
selection  batteries  by  using  those  tests  which  relate  significantly  to 
important  elements  of  individual  jobs.  Thus  the  prediction  battery 
would  vary  in  conjunction  with  variations  in  the  job  elements  present. 

One  obvious  advantage  of  this  procedure  would  be  that  positions  formally 
in  the  same  job  classification  but  having  different  job  elements  could 
have  different  selection  batteries. 

Synthetic  Validity 

Lawshe  (1952)  originally  described  such  a  procedure  which  he  called 
"synthetic  validity."  The  term  was  intended  to  describe  a  process  of 
combining  individual  test-job  element  validities  to  compute  the  estimated 
validity  of  the  selection  battery  for  the  total  job.  However,  predictions 
of  job  suitability  for  individual  personnel  were  based  on  combinations  of 
scores  on  only  those  tests  which  were  most  predictive  of  each  of  the  job 
elements  important  to  the  total  job. 

Lawshe  suggested  that  synthetic  validity  would  be  ideally  suited  for 
validation  research  in  small  companies.  For  these  companies  the  numbers 
of  individuals  in  the  same  jobs  are  usually  so  small  that  empirically 
derived  validity  coefficients  are  not  reliable.  If  pooling  of  data 
across  administrative  units  (to  increase  the  total  N )  is  resorted  to, 
there  is  the  likelihood  that  differences  in  job  content  will  adulterate 
the  actual  validity.  A  situation  which  has  recently  assumed  importance, 
in  which  the  synthetic  validity  technique  might  also  be  useful  is  in 
the  development  of  selection  batteries  for  specific  minority  groups;  the 
Ns  available  for  validation  studies  on  these  groups  are  frequently  very 
small. 

Only  a  few  studies  using  synthetic  validities  have  been  reported 
during  the  20  years  since  the  technique  was  originally  proposed  (Lawshe 
&  Steinberg,  1955;  Guion,  1965;  Drewes,  1961),  and  none  of  these  has 
constituted  a  full-scale  evaluation  of  the  potential  of  the  technique. 

Lawshe  and  Steinberg  found  that  personnel  in  clerical  positions,  requir¬ 
ing  proficiency  in  spelling,  scored  higher  on  a  spelling  test  than  incum¬ 
bents  in  other  jobs.  This  was  interpreted  as  indicating  self  selection 
had  occurred  on  those  jobs  for  clerical  personnel  having  skill  in  spell¬ 
ing.  The  Drewes  and  Guion  studies  produced  evidence  favoring  synthetic 
validity  over  traditional  techniques  of  test  validation.  However,  the 
Drewes  study,  using  subject  matter  of  very  narrow  scope  (psychomotor  tests). 
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and  the  Gu Lon  study,  based  on  an  N  of  L3,  constitute  very  limited  demon¬ 
strations  of  the  technique’s  feasibility. 

Since  estimates  of  overall  synthetic  validity  are  based  on  a  combi¬ 
nation  of  measures  of  test-job  element  relationships,  two  elements  which 
are  critical  for  the  technique  are  accurate  job  analysis  and  accurate 
estimates  of  test-job  element  relationships.  At  the  present  time,  both 
of  these  estimates  are  provided  from  the  judgments  of  experts  and  thus 
the  accuracy  of  individual  estimates  is  largely  dependent  on  the  experi¬ 
ence  and  good  judgment  of  their  originators.  In  addition,  the  usefulness 
of  estimates  is  restricted  by  a  proliferation  of  ad  hoc  job  element  cate¬ 
gorizations,  resulting  in  a  considerable  lack  of  transferability  of  find¬ 
ings.  For  synthetic  validity  to  be  a  technique  of  broad  applicability,  a 
simple,  scientifically  developed  set  of  classification  categories  which 
can  be  applied  by  supervisors  and  non-specialists  in  personnel  is  needed. 

The  Position  Analysis  Questionnaire  (PAQ) 

Such  a  set  of  classification  categories  may  be  provided  in  the  Position 
Analysis  Questionnaire  (PAQ),  which  has  been  developed  from  a  series  of 
studies  of  job  characteristics  by  McCormick  and  his  associates  (McCormick, 
Jeanneret  &  Mecham,  1969a;  McCormick,  Jeanneret  &  Mecham,  1969b;  Mecham  & 
McCormick,  1969a;  Mecham  &  McCormick,  1969b;  Mecham  &  McCormick,  1969c;  and 
McCormick,  Jeanneret  &  Mecham,  1972) .  The  PAQ  was  intended  as  an  empiri¬ 
cally  derived  instrument  for  use  in  job  classification  and  designed  for  use 
in  computing  synthetic  validitities . 

The  questionnaire  was  used  to  collect  job  analysis  data  in  broadspectrum 
studies  of  occupations — some  of  which  covered  536  widely  varied  jobs.  For 
one  study  (McCormick  et  al.,  1969a)  ratings  of  the  relevance  of  68  personal 
attributes  to  each  job  element  characteristic  in  the  PAQ  were  collected  from 
experts.  Separate  principal  component  analyses  of  the  job  analysis  and  the 
personal  attribute  data  developed  from  the  study  produced  factor  structures 
which  were  very  similar.  The  six  major  dimensions  extracted  from  the  per¬ 
sonal  attribute  data  were  the  same  as  six  of  the  seven  major  dimensions 
extracted  from  the  job  analysis  data,  suggesting  that  major  job  dimensions 
had  been  identified. 

These  major  job  dimensions  suggest  a  possible  reason  for  the  low 
validities  typically  found  between  selection  batteries  and  job  performance 
criteria.  The  dimensions  identified  are:  information  input,  mediation 
processes,  work  output,  interpersonal  activities,  work  situation  and  job 
context,  and  miscellaneous.  Of  these,  only  two,  information  input  and 
mediation  processes,  and  parts  of  a  third,  work  output,  appear  to  relate 
to  abilities  which  are  testable  by  means  of  written  tests,  the  usual  method 
of  assessment.  Thus,  based  on  these  job  dimensions,  at  most  only  half  of 
the  elements  defining  job  performance  appear  to  be  predictable  from  written 
test  data.  This  fact,  combined  with  the  less  than  perfect  reliabilities  of 
written  tests,  could  mean  that  the  low  predictor- job  performance  validities 
typically  found  reflect  the  actual  effective  limits  for  predicting  job  per¬ 
formance  from  test  batteries  as  presently  constituted.  However,  it  is  also 
possible  that  the  dimensions  for  which  the  tests  appear  to  be  appropriate 
are  also  the  ones  which  are  either  entirely  or  primarily  responsible  for 
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differentiation  in  level  of  overall  performance  evaluations.  To  examine 
these  possibilities  further  it  is  desirable  to  expand  the  types  of  abil¬ 
ities  measured  in  a  job  selection  battery. 

Computerized  Testing  for  Perceptual  and  Reasoning  Elements 

One  potentially  promising  area  for  additional  types  of  selection  tests 
is  visual  perception.  The  best  predictors  of  job  performance  have  fre¬ 
quently  been  found  to  be  tests  of  clerical  and  perceptual  abilities  (Lucier, 
1958;  Kipnis,  1962;  Phillips,  1966;  Ghiselli,  1966;  and  Curtis,  1971).  In 
addition,  many  of  the  perceptual  elements  among  the  personal  attribute  data 
of  the  studies  of  McCormick,  et  al.  (Mecham  and  McCormick,  1969a,  Appendix 
B)  were  rated  by  classification  experts  as  of  extreme  or  substantial  rele¬ 
vance  to  performance  in  a  wide  variety  of  job  elements. 

For  measuring  certain  visual  perceptual  abilities,  the  use  of  com¬ 
puterized  testing  equipment  seems  to  offer  important  advantages  over 
written  tests.  Some  of  the  features  unique  to  a  computerized  testing 
system  and  not  available  in  regular  paper-and-pencil  tests  are:  exact 
determination  of  stimuli  exposure  times,  moving  stimuli,  exact  control 
of  the  sequence  of  stimuli  presentation,  successive  modification  of  com¬ 
ponents  of  stimuli,  determination  of  response  latencies,  and  sophisticated 
classification  and  scoring  of  responses.  These  characteristics  of  compu¬ 
terized  testing  equipment  also  appear  to  offer  advantages  for  testing 
other  areas  of  ability  identified  in  the  research  of  McCormick  et  al., 
such  as  those  dealing  with  concepts/information  and  visual  short  term 
memory. 

Previous  usage  of  computers  has  included  a  variety  of  tasks  in  the  col¬ 
lection  and  interpretation  of  psychometric  data.  Common  uses  have  been 
interpreting  MMPI  profiles  and  collecting  and  processing  averaged  evoked 
responses  for  the  analysis  of  brain-wave  patterns.  A  number  of  studies 
have  found  that  computerized  training  offered  definite  advantages  over 
traditional  instructor-directed  classroom  training  (Hickey,  1968;  Hansen, 
Dick  &  Lippert,  1968;  Long,  O’Neil  &  Schwartz,  1969;  and  Longo,  1969).  Two 
such  studies  conducted  at  this  Center  (Ford  &  Slough,  1970;  Hurlock,  1971) 
found  A-School  training  to  require  between  40  and  50  percent  less  time  with 
improved  retention  when  computerized  instruction  is  substituted  for  typical 
classroom  instruction.  Students  preferred  the  self-paced  computerized 
instruction  method  by  a  wide  margin. 

The  few  investigations  of  adapting  computers  to  ability  testing  have 
in  the  main  focussed  on  the  potential  of  computerized  branching  techniques 
for  improving  the  efficiency  and  accuracy  of  ability  testing  (Cleary,  Linn, 
&  Rock,  1968;  Linn,  Rock,  &  Cleary,  1969;  Bayroff  &  Seeley,  1967;  Bryson, 
1971).  Dunn,  Lushene,  &  O’Neil  (1971)  used  computerized  testing  to  study 
the  concomitants  of  the  variation  of  latencies  on  the  MMPI.  However,  no 
comprehensive  investigation  of  the  potential  usefulness  of  measures  of 
ability  based  on  utilization  of  the  capabilities  peculiar  to  computerized 
testing  equipment  has  been  found  in  the  literature. 
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The  equipment  available  in  the  Center* s  computer-assisted  instruction 
system  included  computer  consoles,  each  containing  a  cathode  ray  tube 
(CRT)  display  unit,  a  typewriter,  and  a  self-contained  projector-screen 
for  film  presentation.  Subjects  could  indicate  their  answers  to  questions 
on  the  CRT  by  typing  or  by  touching  a  light  pen  target  on  the  screen. 

Research  Objectives 

This  study  was  initiated  to  investigate  a  number  of  questions  concerned 
with  the  empirical  relationships  among  predictors  and  criteria  (defined 
both  globally  and  in  terms  of  individual  job  elements),  the  consistency  of 
such  relationships  within  and  across  ratings,  and  finally,  the  predictive 
accuracy  for  small  samples  of  the  synthetic  validity  technique  in  comparison 
with  the  commonly  used  multiple  linear  regression  technique. 

Data  produced  from  the  study  are  intended  to  provide  information  and 
conclusions  concerning  the  following  topics: 

a.  The  predictive  validities  of  tests  within  three  ratings  for  both 
global  and  specific  job  element  performance  criteria. 

b.  The  consistency  of  the  predictive  relationships  of  tests  and  job 
elements  across  ratings.  Do  tests  which  have  statistically  significant 
relationships  with  a  job  element  in  one  rating  correlate  significantly 
with  the  same  job  element  in  other  ratings? 

c.  Comparison  of  written  and  computerized  tests  designed  to  measure 
the  same  attribute  in  terms  of  (1)  areas  of  measurement,  and  (2)  predic¬ 
tive  validity  for  job  performance  criteria. 

d.  The  accuracy  of  opinions  of  personnel  experts  concerning  the 
types  of  personal  attributes  required  for  the  performance  of  specific 
job  elements. 

e.  Comparison  of  the  predictive  accuracy  of  synthetic  validity  and 
multiple  regression  techniques. 

f.  The  usefulness  of  ratings  of  importance  of  job  elements  and  per¬ 
centage  of  time  spent  on  job  elements  as  moderator  variables  for  the 
validation  of  test  scores  against  global  and  specific  performance  ratings. 


Variables 


PROCEDURE 


Experimental  Tests.  The  attributes  shown  in  the  study  of  McCormick, 
et  al .  (1969a)  which  appeared  to  be  most  relevant  to  the  Navy  ratings 
selected  for  the  study  (Personnelman,  Electrician1 s  Mate,  and  Sonar  Tech¬ 
nician),  and  to  the  specific  characteristics  of  the  computer-assisted 
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testing  equipment  were:  Short  Term  Memory,  Perceptual  Speed,  Perceptual 
Closure,  Movement  Detection,  and  Dealing  with  Concepts/lnforraation.  To 
measure  these  attributes,  eight  computerized  tests  were  constructed, 
supplemented  by  one  auditorially  administered  test,  one  motion  picture 
test,  and  seven  paper-and-pencil  tests  previously  developed  at  this 
Center  or  elsewhere.  To  provide  a  method  of  orientation  for  defining 
the  personal  attributes  measured  by  particular  tests,  wherever  possible, 
the  experimental  battery  for  an  attribute  was  selected  to  include  one  or 
more  factor  reference  tests  for  the  attribute  which  had  been  identified 
in  previous  research.  Most  of  these  reference  tests  were  selected  from 
the  battery  developed  at  Educational  Testing  Service  (ETS)  as  a  result 
of  the  research  of  French,  Ekstrom  &  Price  (1963). 

These  17  experimental  tests  are  described  below.  Samples  of  items 
from  the  eight  computerized  tests  are  shown  in  the  Appendix. 

(1)  Short  Term  Memory 

.  Memory  for  Objects — A  computer-administered  test  consisting  of 
five  frames,  each  frame  containing  pictures  of  from  four  to 
nine  objects  with  simple  one-word  names.  The  set  of  frames, 
arranged  in  ascending  level  of  difficulty,  is  shown  twice 
using  the  same  sequence,  with  exposure  times  approximately  one- 
half  second  per  object  in  a  frame.  For  the  first  showing  sub¬ 
jects  respond  by  typing  the  names  of  the  objects  that  they  can 
remember.  For  the  second  showing,  the  objects  are  identified 
from  word  lists. 

•  Memory  for  Words — A  computerized  test  similar  to  Memory  for 
Objects  in  format,  presentation,  and  exposure  time  for  the 
individual  stimuli,  except  that  words  are  used  instead  of 
pictures  as  in  Memory  for  Objects.  Five  frames  of  three- 
letter  words  and  five  frames  of  five-letter  words,  with  each 
set  of  frames  shown  in  order  of  difficulty,  are  presented. 

Each  three-letter  stimulus  frame  has  a  five-letter  frame 
which  corresponds  to  it  in  number  of  words. 

.  Visual  Memory  for  Numbers  Test — A  computer-administered  test 
composed  of  four-to-13  digit  numbers.  The  stimuli  were  adapted 
from  the  Ms-2  test  (Digit  Span-Visual) ,  which  is  a  factor  ref¬ 
erence  test  for  Memory  Span  in  the  ETS  factor  battery  (French, 
et  al. ) . 

For  some  of  the  numbers,  the  digits  are  flashed  consecutively 
on  the  screen,  one  per  second,  with  answers  to  be  recorded 
after  the  last  digit  of  the  number  has  been  shown.  The  digits 
in  the  other  number  series  are  presented  simultaneously,  in 
sets  of  5  to  13  digits. 
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Auditory  Memory  for  Numbers — A  previously  developed  test  which 
consists  of  aural  presentation  of  number  series  by  tape  record¬ 
ing.  The  four  to  ten  digits  in  each  series  are  read  aloud  at 
about  one  digit  per  second.  Subjects  record  the  digits  they 
recall  after  each  series  has  been  read. 

Object  Number  Test  (Ma-2  of  the  Kit  of  Reference  Tests  of  Cogni¬ 

tive  Factors  [KRT ] ,  French,  et  al.) — A  factor  reference  test  for 
associative  memory.  Subjects  are  given  three  minutes  to  examine 
and  memorize  word-number  pairs;  then,  with  the  words  presented 
in  a  different  order,  the  subjects  must  fill  in  the  matching 
numbers . 

(2)  Perceptual  Speed 

Comparing  Figures — A  computer-administered  test  in  which  the 
frames  are  composed  of  sets  of  squares  or  circles  presented  as 
rows,  vertical  columns,  and  right  and  left  slant  columns.  Three 
to  six  stimulus  pairs  are  shown  on  the  CRT  at  one  time.  Each 
stimulus  has  a  cross  bar  oriented  either  vertically  or  horizon¬ 
tally.  Subjects  are  asked  to  record  as  true-false  answers  whether 
the  cross  bars  of  all  corresponding  pairs  in  the  sets  have  the 
same  orientations  or  some  have  different  orientations.  Two  item 
exposure  modes  are  used.  For  the  first  subtest,  items  are  shown 
only  at  fixed  exposures  consisting  of  approximately  one-third 
second  for  each  stimulus  in  a  set.  Exposures  of  items  in  the 
second  part  are  not  timed,  and  each  stimulus  frame  is  shown  until 
the  subject  records  his  answer. 

Counting  Numbers — This  paper-and-pencil  speed  test  is  an  adapta¬ 
tion  of  Test  4  in  the  Air  Force  Factor  Reference  Battery  II 
(AFPTRC-TN-57-104) .  The  test  involves  scanning  rows  of  numbers 
to  identify  and  count  specified  digits. 

(3)  Perceptual  Closure 

Recognizing  Objects — A  new,  computerized  test  in  which  partially 
blotted  out  pictures  of  common  objects  are  presented.  The  first 
presentation  shows  90  percent  of  the  detail  and  more  detail  is 
added  in  increments  of  10  until  90  percent  of  the  picture  is 
exposed.  Subjects  may  attempt  to  type  the  name  of  the  object 
after  each  stimulus  frame  has  been  presented.  The  measure 
recorded  for  this  test  consisted  of  the  total  number  of  frames 
shown  before  the  objects  were  identified.  Thus,  the  lower  the 
score,  the  better  the  performance. 

Gestalt  Completion  Test  (Cs-1  of  the  KRT) — A  factor  reference 
test  for  Speed  of  Closure.  This  is  a  two-part  written  test  in 
which  partially  blotted-out  pictures  of  objects  are  shown  and 
subjects  must  name  the  objects.  The  stimuli  of  the  test  are 
similar  to  individual  stimulus  frames  of  the  Recognizing  Objects 
test,  but  the  test  does  not  permit  determination  of  recognition 
limens  as  does  the  Recognizing  Objects  test. 
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Concealed  Words  Test  (Cs-2  of  the  KRT) — A  factor  reference  test 
for  Speed  of  Closure.  This  two-part  written  test  consists  of 
common  words  with  parts  of  their  letters  missing.  Subjects 
attempt  to  identify  the  words. 

Hidden  Patterns  Test  (Cf-2  of  the  KRT) — A  factor  reference  test 
for  Flexibility  of  Closure.  This  two-part  written  speed  test 
consists  of  400  patterns,  each  of  which  is  to  be  inspected  to 
determine  if  it  contains  a  test  pattern. 

(4)  Movement  Detection 

Memory  for  Patterns — A  new,  computerized  test  in  which  patterns 
are  formed  by  sequentially  blinking  dots.  Subjects  are  asked 
to  report  whether  or  not  two  consecutive  patterns  are  identical 
or  they  are  asked  to  reproduce  given  patterns. 

Drift  Direction  (Gibson,  1947) — A  motion  picture  test  in  which 
each  item  consists  of  a  dot  moving  slowly  next  to  a  line.  Sub¬ 
jects  must  tell  whether  the  dot  moves  away  from,  toward,  or 
parallel  to  the  line. 

(5)  Dealing  with  Concepts/Information 

Twelve  Questions — This  computer-administered  test  is  similar  to 
the  Twenty  Questions  game  in  that  subjects  are  asked  to  guess 
the  name  of  an  object  based  on  yes-no  answers  supplied  by  the 
computer  to  questions.  It  differs  from  Twenty  Questions  in 
that  the  questions  are  supplied  in  the  test  rather  than  being 
posed  by  the  subject.  The  subject's  objectives  are  to  select 
those  questions  that  will  provide  the  quickest  identification 
of  the  object  and  to  avoid  questions  which  are  redundant  or 
useless.  As  selecting  questions  and  guessing  the  name  of  the 
object  proceed,  questions  more  specific  to  the  object  are  given. 

Password — This  computer-administered  test  resembles  the  regular 
"Password11  game  in  that  sets  of  words  are  shown  which  suggest  a 
target  word.  Five  separate  words  are  shown  as  clues.  After  the 
first  two  clues  and  each  succeeding  one,  subjects  may  attempt  to 
identify  the  object  by  typing  its  name  on  the  keyboard. 

Nonsense  Syllogisms  Test  (Rs-1  of  the  KRT) — A  factor  reference 
test  for  Syllogistic  Reasoning.  The  items  in  this  two-part 
written  test  consist  of  formal  syllogisms  in  which  the  terms 
are  nonsense  words.  Subjects  must  indicate  whether  the  conclu¬ 
sions  shown  are  correct  or  incorrect. 

Inference  Test  (Rs-3  of  the  KRT) — A  factor  reference  test  for 
Syllogistic  Reasoning.  This  two-part  written  test  consists  of 
40  items,  each  containing  a  statement  and  five  possible  conclu¬ 
sions  which  might  be  drawn  from  the  statement.  Subjects  indi¬ 
cate  whether  or  not  the  conclusions  are  logically  justified. 
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Operational  Tests  and  Biographical  Variables 


•  Armed  Forces  Qualification  Test  (AFQT) — A  measure  of  vocabulary, 
arithmetic  reasoning,  spatial  reasoning,  and  mechanical  knowledge. 
Scores  are  recorded  as  percentiles  for  a  large  sample  of  eligibles 
for  military  service  and  range  from  10  to  99. 

Scores  of  the  following  classification  tests  are  reported  as  Navy 
Standard  Scores  and  have  means  for  an  unrestricted  Navy  sample  of  about 
50  and  standard  deviations  of  about  10. 

.  General  Classification  Test  (GCT) — A  test  involving  ability  to 
comprehend  and  define  words  and  to  reason  verbally. 

.  Arithmetic  Reasoning  Test  (ARI) — A  test  of  quantitative  aptitude 
involving  mathematical  reasoning  and  problem  solving. 

.  Mechanical  Test  (MECH) — A  test  of  basic  mechanical  and  electrical 
knowledge  and  comprehension  of  mechanical  principles  and  relation¬ 
ships  . 

.  Clerical  Test  (CLER) — A  test  of  perceptual  speed  and  accuracy 
which  involves  checking  whether  pairs  of  numbers  are  the  same 
or  different. 

•  Sonar  Pitch  Memory  Test  (SONAR) — A  test  of  ability  to  detect 
and  remember  small  differences  in  tonal  pitches. 

.  Radio  Code  Aptitude  Test  (RADIO) — A  test,  administered  by  phono¬ 
graphic  recording,  measuring  ability  to  learn  International  Morse 
Code . 

.  Electronic  Technician  Selection  Test  (ETST) — A  test  of  knowledge 
of  mathematics,  science,  electricity,  and  electronics. 

.  Shop  Practices  (SHOP) — A  test  of  knowledge  of  the  use  of  tools 
and  other  shop  equipment. 

Also  extracted  from  the  military  records  for  use  as  biographical  vari¬ 
ables  were  Years  of  Education  and  Year  of  Birth. 

Sample 

The  experimental  battery  was  administered  to  a  total  of  385  students 
at  the  Naval  Training  Center  (NTC) ,  San  Diego  during  May  and  June  of 
1972.  Subjects  were  chosen  from  personnel  in  the  first  two  weeks  of  A- 
School  for  three  ratings  having  widely  varied  duties.  Also  tested,  in 
order  to  increase  the  sample  sizes  of  the  two  ratings  having  relatively 
small  /V's,  were  recruits  in  their  final  week  of  training,  who  were  eli¬ 
gible  for  A-School  but  had  not  yet  received  post-recruit  assignments. 

The  sample  contained  the  following  rating  group  subsamples: 


9 


A-School  students  in  first  two  weeks  of  training; 


Personnel  Man 
Electrician’s  Mate 
Sonar  Technician 


82 

107 

58 


138 


385 


It  was  anticipated  that  some  of  the  138  subjects  in  the  recruit  sample 
would  receive  assignments  for  training  in  one  of  the  three  ratings  and 
could  thus  be  added  to  the  sample  during  the  on-job  validation  phase. 
Meanwhile  the  additional  recruits  tested  would  produce  more  stable 
statistics  on  the  experimental  tests. 


Analysis 


Intercorrelations  of  the  operational  and  experimental  tests  and  the 
biographical  variables  were  computed  and  correlations  of  tests  intended 
to  measure  the  same  attribute  were  compared.  Particular  attention  was 
given  to  the  relationships  between  computer-administered  and  factor- 
reference  tests. 

To  gain  further  perspective  concerning  interrelationships  in  the  pre¬ 
dictive  battery,  a  principal  components  analysis  was  performed,  using  l’s 
in  the  diagonal.  By  limiting  the  factors  extracted  to  those  having 
eigenvalues  greater  than  one,  seven  principal  components  were  extracted. 
They  were  rotated  by  the  varimax  method. 

Correlations .  That  the  sample  was  highly  restricted  may  be  seen 
from  the  means  and  standard  deviations  of  the  operational  tests,  as 
shown  in  Table  1.  As  expected,  the  sample  had  larger  means  and  smaller 
standard  deviations  than  typical  samples  of  Navy  enlisted  recruit  input. 
The  means  of  the  classification  tests  ranged  between  a  third  and  a  whole 
standard  deviation  above  50,  the  general  mean,  and  the  standard  devia¬ 
tions  ranged  from  70  to  90  percent  of  the  usual  size.  These  restric¬ 
tions  resulted  in  intercorrelations  among  the  operational  tests  which 
were  20  to  25  points  lower  than  those  usually  computed  for  unrestricted 
samples . 

A  large  number  of  the  correlations  among  the  predictor  variables  are 
statistically  significant.  Significant  intercorrelations  are  particu¬ 
larly  common  among  the  operational  tests.  The  computerized  tests  also 
tend  to  show  significant  correlation  with  each  other. 

The  correlations  in  Table  1  indicate  the  extent  to  which  the  com¬ 
puterized  tests  overlap  the  reference  tests  for  the  attribute  they  were 
both  designed  to  measure. 
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TABU  I 

Descriptive  Statistics  Tor  Operntiott.it  Aiul  I. xpcrinent.il  Variables 


1 

2 

3 

4 

5 

6 

7 

8 

9  10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

:1 

22 

23 

24 

25 

26 

27 

28 

29 

Mean 

S.  II. 

N 

1.  AFQ1 

5JB 

50 

05 

H 

22 

40  -08 

36 

50 

11 

\T_ 

04 

^9 

36 

11 

11 

28 

-02 

07 

11 

22 

22 

22 

2£ 

22  -is 

72.39 

16.00 

320* 

2.  GCT 

- 

AA 

32 

09 

28 

11 

50 

30  -08 

27 

04 

27 

02 

06 

30 

58 

23 

03 

11 

03 

05 

32 

30 

22 

27 

40 

22  -27 

60.78 

6.20 

319 

3.  ARi 

~ 

13 

26 

08 

34 

11 

06  -14 

29 

11 

03 

_16 

17_ 

22 

40 

20 

10 

26 

04 

10 

11 

22 

23 

22 

H 

11 

00 

56.24 

6.66 

319 

4.  MiOl 

-07 

20 

08 

35 

69  -05 

20 

-05 

_19 

-01 

-n 

14 

11 

-03 

11 

07 

-01 

00 

09 

01 

-03 

09 

06 

oo  -22 

53.25 

6.90 

319 

5.  CLCR 

- 

-05 

20 

08 

-12  -2± 

26 

48 

10 

28 

12 

08 

11 

07 

06 

37 

H 

11 

00 

OS 

OS 

17 

03 

00 

-05 

54.88 

8.60 

319 

1.  SONAR 

~ 

li 

ii 

^6  02 

IS^ 

02 

T3 

06 

03 

07 

T3 

J_4 

10 

22 

08 

07 

22 

2£ 

25 

11 

11 

22  -n9 

59  46 

9.25 

317 

7.  RADIO 

26 

«o 

o 

K> 

O 

11 

24 

05 

11 

08 

12 

11 

2^3 

15 

27 

22 

23 

08 

11 

26 

22 

li 

20  -04 

59.74 

8.  99 

313 

H.  RTST 

- 

36  -10 

11 

09 

13 

04 

02 

25 

11 

08 

11 

11 

00 

12 

11 

11 

09 

37 

22 

11  'll 

59.54 

6.74 

313 

V.  SHOP 

- 12 

08 

-04 

04 

’ll 

-19 

03 

17 

-04 

02 

02 

00 

-02 

11 

01 

12 

10 

10 

-05 

-10 

54.60 

6.78 

313 

ID.  Year  of  Mrth 

~ 

'll 

'll 

01 

-07 

-03 

-08 

-13 

“11 

-07 

-13 

■11 

-04 

-10 

01 

-22 

-05 

-04 

10  04 

52.26 

1.56 

373 

1 1  Hidden  Pat  terns 

- 

30 

32 

29 

10 

1_4 

22 

14 

19 

28 

14 

22 

11 

u 

17 

29 

26 

22  -21 

63.91 

2D.  05 

.*ul 

.  ,  Count lng 

Nuwbra 

- 

02 

22 

05 

02 

13 

10 

06 

27 

li 

10 

01 

22 

26 

22 

0? 

04 

06 

30.00 

b.  25 

370 

j,  Gestalt 

Couplet  ion 

- 

29 

05 

09 

12 

05 

J_3 

21 

-01 

10 

10 

22 

04 

Of' 

18 

22  £4 

16.42 

3.31 

360 

14.  Concealed  Words 

- 

08 

it 

11 

_M 

10 

11 

09 

21 

03 

22 

08 

06 

02 

11 

-II 

23.39 

5.64 

369 

iS.  Object  Niraher 

- 

11 

20 

11 

05 

10 

04 

16 

12 

10 

11 

03 

08 

09 

05 

12.48 

6.66 

370 

.  Non  a  in  se 

Sy 1  log  1 sea 

- 

11 

11 

09 

12 

00 

09 

22 

10 

il 

08 

21 

06 

-12 

5.43 

6.84 

36<> 

17.  Inferenre 

-- 

U 

08 

r7 

05 

10 

28 

22 

22 

11 

i! 

22  -22 

11.28 

4.38 

369 

.„  Aud.  Memory 
for  Nuabcra 

" 

06 

13 

02 

13 

10 

32 

52 

10 

11 

23  -os 

117.45 

14.05 

34  7 

Hi.  Drift  Direct.  Ion 

- 

12 

01 

11 

03 

10 

22 

22 

10 

11 

’ll 

13.  13 

7.41 

368 

Coaparlna 

Figures,  Spd.n 

- 

33 

07 

U 

26 

24 

27 

11 

09  *22 

34.02 

8.39 

37.3 

Cotspar  ing 


1 1  Figures,  Un- 
al rue  lured* 

-  04  OS 

22 

16 

13 

2£ 

2f  -i4 

13.92 

2.54 

373 

Memory  for 

Patterns,  T-M 

--  00 

22 

11 

24 

04 

10  -04 

5.  13 

.3.58 

367 

Twelve 
hirut  Iona* 

- 

li 

11 

22 

33 

22  -T3 

23.90 

28.00 

.336 

.4  Mwsorv  for 
'  Words  1 

-- 

40 

26 

11 

50  -25 

32.09 

8.79 

373 

Via.  Mrmorv 
for  Numbers* 

- 

24 

25 

21  -11 

82.46 

17.38 

373 

Meaory  for 

26.  Patterns, 

Free  Response* 

- 

20 

24  -28 

16.5.3 

8.03 

.367 

27  Password* 

- 

24  -22 

22.24 

5.72 

358 

.  Hrssirv  for 
*  ’  Objects* 

--  -19 

38.  22 

K.  96 

373 

Hecngnlt In* 

*  Objects  * 

- 

IS.  9'. 

4.13 

373 

Note.  * 

1.  Dctlasl  points  have  been  oaitted  froa  the  correlation  coefficients 

2.  Coefficients  significant  at  p  <  .05  and  p  «  .01  have  been  Identified  by  single  ( — )  and  double  l=)  underlines,  respectively. 
iCnaputiT'  administered  testa. 
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(1)  Short  Term  Memory 


The  visual  and  auditory  Memory  for  Numbers  Tests  are  well  suited 
as  reference  tests  for  the  Memory  Span  factor.  The  Object-Number  Test 
has  been  shown  to  measure  a  somewhat  different  ability,  associative 
memory. 


Intercorrelations  among  the  four  short  term  memory  tests  (vari¬ 
ables  18,  24,  25,  and  28)  ranged  from  .20  to  .50,  with  Memory  for  Words 
having  the  highest  intercorrelations  with  the  other  tests.  As  would  be 
expected  from  the  factor  structures  determined  in  previous  studies 
(French,  et  al. ,  p.  22)  correlations  of  the  Object-Number  Test  with  the 
Memory  Span  tests  were  considerably  lower  and  were  barely  significant. 
Correlation  of  the  two  memory  for  numbers  tests,  .52,  is  an  estimate  of 
the  association  between  tests  which  measure  the  same  factor. 

(2)  Perceptual  Speed 

The  Counting  Numbers  test  from  the  Air  Force  Factor  Reference 
Battery  correlated  .48  with  CLER,  confirming  that  CLER  is  a  measure  of 
perceptual  speed.  The  lower  correlation  of  Comparing  Figures  with 
Counting  Numbers  (.27)  indicates  that  the  computer-administered  test  is 
not  as  pure  a  measure  of  perceptual  speed  as  CLER. 

(3)  Perceptual  Closure 

The  three  paper-and-pencil  reference  tests  for  perceptual  closure 
intercorrelated  about  .30  with  one  another.  Correlations  of  Recognizing 
Objects  with  these  tests,  in  general,  were  much  lower.  Surprisingly, 
Recognizing  Objects  correlated  only  .24  with  Gestalt  Completion,  a  test 
which  it  appears  to  closely  resemble,  and  its  correlations  with  Hidden 
Patterns  and  Concealed  Words  were  barely  significant.  This  indicates 
that  the  differences  between  the  computer  and  the  paper-and-pencil  modes 
for  tests  of  perceptual  closure  are  greater  than  they  appear  to  be. 

(4)  Mo vemen t  Detection 


Because  of  the  nature  of  the  Movement  Detection  abilities,  paper- 
and-pencil  factor  reference  tests  were  not  available.  Furthermore,  the 
experimental  tests  for  this  attribute  have  little  in  common.  It  is  clear 
from  the  low  correlation  of  Memory  for  Patterns  and  Drift  Direction  (which 
barely  approached  significance)  and  the  differences  between  the  correla¬ 
tions  of  these  tests  with  others  in  the  battery  that  the  two  tests  are 
not  measuring  the  same  ability. 

(5)  Dealing  with  Concepts/Information 

Although  reference  tests  were  not  available  for  this  attribute 
as  such,  Nonsense  Syllogisms  and  Inference  are  reference  tests  for 
syllogistic  reasoning,  a  similar  type  of  ability. 
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Dealing  with  Concepts/ Inf ormat ion  is  apparently  less  focused  and 
more  diffuse  than  the  first  three  attributes.  Intercorrelations  among 
the  Inference,  Twelve  Questions  and  Password  tests  were  about  .30.  Cor¬ 
relations  of  Twelve  Questions  and  Password  with  Nonsense  Syllogisms  were 
about  ten  to  fifteen  points  lower. 

Although  correlations  among  most  of  the  tests  for  particular  attri¬ 
butes  were  significant  and  substantial,  the  communalities  were  generally 
low.  Even  for  reference  tests  for  the  same  factor,  the  uncorrelated 
variance  was  larger  than  the  covariance. 

Factor  Loadings.  Rotated  loadings  for  the  seven  principal  compon¬ 
ents  derived  from  26  operational  and  experimental  variables  are  shown  in 
Table  2.1  In  Table  2  loadings  of  about  .30  or  greater  were  considered 
significant. 

These  factors  are  no  doubt  somewhat  distorted  from  those  derivable 
from  a  full  range  sample.  An  obvious  distortion  is  a  "gu  factor  which 
is  underdefined  as  a  result  of  the  low  intercorrelations  among  opera¬ 
tional  tests,  mentioned  in  the  Correlations  section.  However,  many  of 
the  relationships  emerging  from  the  factoring  are  consistent  with  past 
findings.  Therefore,  the  factor  structure  of  the  battery  may  be  gener¬ 
ally  applicable  to  full  range  samples. 

(1)  Factor  1 


With  the  AFQT,  GCT,  ARI,  Nonsense  Syllables,  and  Inference  tests 
loading  heavily  on  this  factor,  it  clearly  measures  the  broad-based  cog¬ 
nitive  ability  originally  identified  by  Spearman  as  "g."  It  is  consist¬ 
ent  that  Object  Number,  a  reference  test  for  associative  or  rote  learning, 
should  load  positively  on  this  basic  academic  ability. 

(2)  Factor  2 


The  heavy  loadings  of  CLER  and  Counting  Numbers  and  the  absence 
of  significant  loadings  for  these  tests  on  other  factors  indicate  that 
Factor  2  is  a  measure  of  perceptual  speed.  Comparing  Figures  loaded 
significantly  but  less  heavily  on  this  factor,  serving  as  an  additional 
indication  that  the  test  is  less  pure  as  a  measure  of  perceptual  speed 
than  CLER.  Whether  the  non-perceptual  speed  elements  in  Comparing 
Figures  will  prove  of  value  for  predicting  on-job  performance  is  to  be 
determined  in  the  follow-up  portion  of  the  study. 


^Preliminary  factoring  of  the  complete  battery  of  operational  and 
experimental  tests  suggested  that  inclusion  of  all  of  the  tests  would 
serve  to  distort  the  interrelationships  between  tests  and  principal 
components.  Therefore,  elimination  of  the  Auditory  Memory  for  Numbers, 
Comparing  Figures,  Unstructured  and  Memory  for  Patterns,  T-F  tests  from 
the  final  factoring  was  done  to  clarify  and  to  more  accurately  present 
the  factor  structure. 
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TABLE  2 

Rotated  Factor  Loadings  of  Twenty-Six  Operational 
and  Experimental  Variables 
(N  =  373) 


Variable 

Factor 

1 

2 

3 

4 

5 

6 

7 

h2 

AFQT 

5640 

0097 

0533 

4675 

3108 

-0160 

3027 

73 

GCT 

6628 

0157 

2438 

3046 

0856 

3192 

0575 

70 

ARI 

7112 

2989 

0324 

0054 

-0245 

-0978 

2510 

67 

MECH 

1940 

-0856 

-0404 

7907 

2077 

-0062 

1269 

73 

CLER 

1683 

7418 

-0060 

-1544 

1489 

-0670 

-1185 

64 

SONAR 

0271 

-0570 

0815 

2285 

1066 

1270 

6289 

48 

RADIO 

1962 

3542 

1604 

-0107 

0322 

-1660 

6051 

58 

ETST 

5784 

1396 

2712 

4377 

-0234 

-0447 

1088 

63 

SHOP 

1133 

-0441 

-0770 

8351 

-0876 

1831 

0510 

76 

Year  of  Birth 

-0003 

-4631 

3828 

-1587 

0289 

-3589 

-0619 

52 

Hidden  Patterns 

1825 

4300 

0050 

1441 

4673 

1355 

2072 

52 

Counting  Numbers 

0221 

7395 

0545 

-0661 

0341 

-0195 

0540 

56 

Gestalt  Completion 

0561 

-0052 

1502 

1053 

7669 

1737 

-0603 

66 

Concealed  Words 

1083 

2473 

0272 

-2018 

6482 

-1332 

0553 

56 

Object  Number 

3343 

-0225 

-2096 

-4606 

1136 

2216 

2084 

47 

Nonsense  Syllogisms 

5488 

-0050 

-0426 

-0535 

1572 

1553 

-0452 

36 

Inference 

6966 

1371 

1197 

0439 

-0146 

3340 

-0258 

63 

Drift  Direction 

0123 

0620 

0334 

1100 

3610 

-0260 

3107 

24 

Comparing  Figures3 

0730 

5451 

1085 

0396 

1506 

1370 

1942 

40 

Twelve  Questions 

2034 

-0118 

0669 

0569 

0104 

6638 

1047 

50 

Memory  for  Words3 

1448 

0298 

5643 

-1273 

0589 

2640 

3777 

57 

a 

Visual  Memory  for  Numbers 

0569 

1738 

1290 

-1220 

-0536 

3995 

5753 

56 

Memory  for  Patterns 

1503 

3876 

5416 

1620 

-1090 

0656 

1877 

54 

Password3 

1707 

0675 

2982 

0176 

0888 

6242 

0429 

52 

Memory  for  Pictures 

0100 

-1054 

6060 

-1341 

1468 

0941 

3416 

55 

Recognizing  Objects3 

-0046 

-1287 

-5796 

-2288 

-2526 

-2587 

2092 

58 

Eigenvalue 

5.33 

2.41 

1.81 

1.48 

1.34 

1.23 

1.08 

Percentage  of  Variance 

20.50 

9.28  6.97  5.68 

(Total  Percentage 

5.18  4.72 

=  56.50) 

4.17 

Note . — 

1.  Decimal  points  are  omitted  from  the  factor  loadings  and  the  communalities . 

2.  Factor  loadings  ^  > | . 3000 1  are  underlined. 

3.  Small  percentages  of  missing  data  were  present  for  some  of  the  operational 
variables . 

aComputer-administered  test. 
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Speed  of  performance,  which  is  important  for  both  ARI  and  Hidden 
Patterns,  may  be  the  reason  these  tests  load  significantly  on  Factor  2. 
The  precise  and  rapid  perceptual  processes  required  for  RADIO  may  be  the 
reason  for  inclusion  of  this  test  in  the  factor.  For  this  sample,  age 
was  associated  positively  with  perceptual  speed,  a  finding  also  reported 
in  previous  research  (Cory,  1971,  p.  11). 

(3)  Factor  3 

Factor  3,  as  well  as  Factor  7,  measures  a  short  term  memory 
ability.  However,  the  abilities  are  primarily  distinguished  by  their 
degree  of  associat ional  content.  Thus  the  memory  tests  loading  most 
heavily  on  Factor  3  require  the  direct  recall  of  stimuli  having  high 
associat ional  values  or  substantial  verbal  mediation — i.e.,  words,  pat¬ 
terns,  and  the  names  of  objects.  Interestingly,  Recognizing  Objects  has 
its  only  significant  loading  on  this  factor  rather  than  on  perceptual 
closure,  suggesting  a  possible  close  association  between  memory  and 
closure.  Factor  3  was  negatively  associated  with  age. 

(4)  Factor  4 


This  factor,  with  MECH  and  SHOP  having  the  highest  loadings,  pri¬ 
marily  represents  mechanical  reasoning  and  tool  knowledge.  Thus  it  con¬ 
sists  of  a  crystallized  ability  having  primarily  practical  rather  than 
academic  content.  However,  measures  of  more  academic  crystallized  abili¬ 
ties,  AFQT ,  ETST,  and  GCT,  also  load  significantly  on  it. 

Factor  4  is  the  second  crystallized  ability  factor  on  which  the 
Object-Number  test — clearly  a  measure  of  fluid  ability — loaded  signifi¬ 
cantly.  However,  the  Object-Number  test  was  negatively  associated  with 
Factor  4,  in  contrast  to  the  positive  loadings  of  the  test  on  Factor  1, 
a  generalized  factor  of  academic  aptitude. 

(5)  Factor  5 


The  heavy  loadings  of  Gestalt  Completion,  Concealed  Words,  and 
Hidden  Patterns  on  Factor  5  indicate  it  to  be  a  measure  of  perceptual 
closure.  Drift  Direction  and  AFQT  probably  loaded  significantly  on  this 
factor  by  virtue  of  their  visual  elements. 

This  analysis  confirms  the  diversity  of  focus  of  the  two  tests 
selected  for  Movement  Detection,  since  the  tests  loaded  on  different 
principal  components.  Memory  for  Patterns  loaded  on  Perceptual  Speed 
and  Memory  for  high  Associative  Stimuli  while  Drift  Direction  loaded 
on  Perceptual  Closure  and  Memory  for  low  Associative  Stimuli. 

( 6 )  Factor  6 

This  sequential  reasoning  factor  is  primarily  defined  by  computer- 
administered  tests,  with  the  tests  loading  most  heavily  being  Password  and 
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Twelve  Questions*  It  is  somewhat  similar  to  the  factor  identified  as 
Serial  Integration  by  Siebert  &  Snow  (1965,  p.  61),  who  used  a  motion 
picture  rather  than  a  computer  terminal  mode  of  administration.  Factor 
6  involves  primarily  serial,  deductive  reasoning  requiring  frequent  up¬ 
dates  of  the  information  pool.  The  high  verbal  content  of  GCT  apparently 
accounts  for  its  significant  loading  on  this  factor.  Age  is  positively 
related  to  performance  on  Factor  6. 

(7)  Factor  7 


This  short  term  memory  factor  is  primarily  defined  by  stimuli 
requiring  rote  reproduction,  with  little  if  any  associational  content. 
Thus  SONAR,  RADIO,  and  Visual  Memory  for  Numbers,  requiring  as  they  do 
reproduction  of  musical  pitches,  sound  patterns,  and  numeric  digits  have 
the  heaviest  loadings  on  Factor  7,  and  tests  involving  more  associative 
stimuli  have  lower  loadings.  Perceptual  elements  apparently  account  for 
the  significant  loadings  of  Drift  Direction  and  AFQT  on  this  factor. 


DISCUSSION 


It  is  clear  that  the  experimental  battery  represents  a  considerable 
increase  in  the  breadth  of  abilities  covered  beyond  those  in  the  opera¬ 
tional  battery.  Three  of  the  seven  factors  were  defined  wholly  or 
principally  by  experimental  tests.  For  two  of  these  factors,  3  and  6, 
computerized  administration  appears  to  represent  the  difference  between 
measuring  and  not  measuring  the  ability. 

Computer  administration  of  the  Memory  for  Numbers  Test  offers  advan¬ 
tages  over  written  but  not  over  aural  administration  by  a  monitor.  Com¬ 
puterized  administration  of  tests  of  perceptual  speed  does  not  appear  to 
offer  any  advantage  over  the  traditional  paper-and-pencil  measures  for 
this  attribute. 

Results  from  the  Recognizing  Objects  test  suggest  that  memory  ele¬ 
ments  may  have  important  influences  on  the  recognition  threshold  for 
indistinct  visual  stimuli.  To  clarify  these  relationships  additional 
studies  are  needed  in  which  computer-administered  and  written  tests  of 
perceptual  closure  can  be  compared. 

Although  short  term  memory  tests  were  the  major  defining  elements  in 
two  factors,  in  general  the  study  was  in  agreement  with  the  findings  of 
Christal  (1958)  and  Thurstone  (1946),  among  others,  in  failing  to  find 
clear  cut  evidence  for  memory  factors  defined  by  contents  specific  to 
the  data.  The  different  memory  abilities  which  emerged,  corresponded 
with  the  high  and  low  associational  characteristics  of  the  stimuli. 
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Collection  of  on-job  performance  marks  from  supervisors  of  the  sub¬ 
jects  eight  to  ten  months  after  their  completion  of  A-School  has  been 
completed  and  is  now  being  processed.  A  report  will  shortly  be  prepared 
covering  the  additional  topics  mentioned  in  the  Research  Objectives  sec¬ 
tion  . 
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Am  Nit  IX 


ILLUNTKA1  I VI:  ITIiMS  I  ROM  Till.  1  1  f  .1 1 1  COMPUTER  I  .Hill  TESTS 


1.  MEMORY  I'OR  OMIiCTS 


MEMORY  mu  WORDS 


PRIZE  IVORY  TABLE 
8T0VE  MUSIC  SOLID 


3.  VISUM.  MEMORY  R)R  NUMBERS  TEST 
2  5  16* 

I  2 4!» 5038 7* 


'I.  COMPARING  IT  CURES 
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APPENDIX  (Continued) 


6.  MEMORY  FOR  PATTERNS 


7.  COMPUTERIZED  12  QUESTIONS 
Mineral 

Frequently  larger  than  a  glove 

1.  Is  it  often  used  as  clothing? 

2.  Is  it  made  of  a  soft  material? 

3.  Is  it  often  used  at  meals? 

4.  Do  people  often  wear  it? 

5.  Does  it  have  moving  parts? 

6.  Does  it  have  a  hard  surface? 

7.  Is  it  always  found  on  an  auto? 

8.  Is  it  made  at  least  partly  of  glass? 

9.  Does  it  have  more  than  one  use? 

10.  Does  it  use  electricity? 


Mirror 


11.  Is  it  sometimes  used  by  magicians? 

12.  Do  men  and  women  use  it  equally  often? 

13.  Is  it  often  used  before  a  person  goes  out? 

14.  Can  one  use  it  with  his  eyes  closed? 

15.  Must  one  touch  it  to  use  it? 

16.  Does  it  appear  dark  in  the  light? 

17.  Can  it  be  used  to  send  messages? 

18.  Can  it  improve  one’s  appearance? 


8.  COMPUTERIZED  PASSWORD 


Metal 

Finger 

Circle 

Shiny 

Soaring 

Emblem 

Feathers 

Large 

Wedding 


Bald 
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